Target object identifying device, target object identifying method and target object identifying program

ABSTRACT

Monitoring target matching means 71 matches monitoring targets shown in video captured by one or more imaging devices, and identifies monitoring targets estimated to be the same monitoring target, as an identified monitoring target. Target object identifying means 72 identifies a desired target object from one or more identified monitoring targets captured, using imaging times of each of the one or more identified monitoring targets.

The present application is a Continuation application of Ser. No.14/765,621 filed on Aug. 4, 2015, which is a National Stage Entry ofPCT/JP2014/000523 filed on Jan. 31, 2014, which claims priority fromJapanese Patent Application 2013-072178 filed on Mar. 29, 2013, thecontents of all of which are incorporated herein by reference, in theirentirety.

TECHNICAL FIELD

The present invention relates to a target object identifying device,target object identifying method, and target object identifying programfor identifying a desired target object from monitoring targets.

BACKGROUND ART

Surveillance cameras are installed in stations, specific facilities, andthe like, and video captured by the surveillance cameras is analyzed toperform various determination. As an example, a person or object presentin a monitoring range for an unusually long time is identified as asuspicious person or object.

As a relevant technique, a behavior analysis method of tracking aspecific person and analyzing his or her behavior is known. In thebehavior analysis method, for example, where the specific person ispresent is recognized using one camera or a plurality of cameras thatoverlap in monitoring area, and the temporal changes of the position ofthe person are tracked to determine where and how long the personstayed.

A person recognition method of performing face matching to recognize aspecific person from a captured image is known, too. Patent Literature(PTL) 1 describes a face image recognition device capable of a fasterface image recognition process to simplify registration operation. Theface image recognition device described in PTL 1 registers a frontalface image of each person to be recognized and an average face image ofthe person in an orientation other than the front, and matches thefeatures of a face area extracted from video against the registered faceimages to recognize the face image in the video.

CITATION LIST Patent Literature(s)

PTL 1: Japanese Patent Application Laid-Open No. 2012-238111

SUMMARY OF INVENTION Technical Problem

With the above-mentioned behavior analysis method, the range in whichthe behavior of the person can be analyzed depends on the imaging rangeof the camera(s). For example, in the case where one camera is used, therange in which the behavior of the person can be analyzed is limited tothe range that can be captured by the camera. To cover many ranges, alot of cameras need to be used with no gap between the monitoringranges. Besides, when capturing the person in a crowded situation, theperson is often unable to be seen with other people in the way, and thusit is difficult to completely track the specific person.

In the case of using the face image recognition device described in PTL1, information of any suspicious person to be found needs to be providedbeforehand. In other words, the face image recognition device describedin PTL 1 cannot be used in a situation where who is suspicious isunknown.

The present invention accordingly has an object of providing a targetobject identifying device, target object identifying method, and targetobject identifying program capable of, even in the case where themonitoring range is wide or crowded, identifying a target object presentin the range for an unusually long time from monitoring targets.

Solution to Problem

A target object identifying device according to the present inventionincludes: monitoring target matching means which matches monitoringtargets shown in video captured by one or more imaging devices, andidentifies monitoring targets estimated to be the same monitoringtarget, as an identified monitoring target; and target objectidentifying means which identifies a desired target object from one ormore identified monitoring targets, using imaging times of each of theone or more identified monitoring targets.

A target object identifying method according to the present inventionincludes: matching monitoring targets shown in video captured by one ormore imaging devices, and identifying monitoring targets estimated to bethe same monitoring target, as an identified monitoring target; andidentifying a desired target object from one or more identifiedmonitoring targets, using imaging times of each of the one or moreidentified monitoring targets.

A target object identifying program according to the present inventioncauses a computer to execute: a monitoring target matching process ofmatching monitoring targets shown in video captured by one or moreimaging devices, and identifying monitoring targets estimated to be thesame monitoring target, as an identified monitoring target; and a targetobject identifying process of identifying a desired target object fromone or more identified monitoring targets, using imaging times of eachof the one or more identified monitoring targets.

Advantageous Effects of Invention

According to the present invention, even in the case where themonitoring range is wide or crowded, a target object present in therange for an unusually long time can be identified from monitoringtargets.

BRIEF DESCRIPTION OF DRAWINGS

FIG. 1 It is a block diagram depicting a structural example of anexemplary embodiment of a target object identifying device according tothe present invention.

FIG. 2 It is an explanatory diagram depicting an example of informationstored in a person identification information storage unit 18.

FIG. 3 It is an explanatory diagram depicting an example of informationstored in a person information storage unit 19.

FIG. 4 It is an explanatory diagram depicting another example ofinformation stored in the person information storage unit 19.

FIG. 5 It is an explanatory diagram depicting an example of theoperation of determining the stay time of each monitoring target.

FIG. 6 It is an explanatory diagram depicting an example ofdetermination conditions for specifying time widths.

FIG. 7 It is an explanatory diagram depicting a specific example ofinformation stored in the person information storage unit 19.

FIG. 8 It is an explanatory diagram depicting an example of theoperation of capturing a monitoring target by a plurality of videoacquisition units 11.

FIG. 9 It is a flowchart depicting an example of the operation of thetarget object identifying device.

FIG. 10 It is a block diagram schematically depicting the target objectidentifying device according to the present invention.

FIG. 11 It is another block diagram schematically depicting the targetobject identifying device according to the present invention.

DESCRIPTION OF EMBODIMENT(S)

The following describes an exemplary embodiment of the present inventionwith reference to drawings. In the following description, the case wherepersons as a typical example of monitoring targets are monitored and anyperson staying in the monitoring range for an unusually long time isidentified as a target object is used as an example. The monitoringtargets are, however, not limited to persons. For example, themonitoring targets may be objects such as cars. In this case, any carparked in the monitoring range for a long time is identified as a targetobject. In the present invention, the term “target object” includes notonly an object but also a person.

FIG. 1 is a block diagram depicting a structural example of an exemplaryembodiment of a target object identifying device according to thepresent invention. The target object identifying device in thisexemplary embodiment includes a video acquisition unit 11, a personidentification information analysis unit 12, a person identificationinformation management unit 13, a long stayer estimation unit 14, aperson identification information matching unit 15, a person informationmanagement unit 16, a result output unit 17, a person identificationinformation storage unit 18, and a person information storage unit 19.

The video acquisition unit 11 acquires video of a predeterminedmonitoring range. The video acquisition unit 11 also acquires the time(hereafter referred to as “imaging time”) at which the video wasacquired. The video acquisition unit 11 outputs the video of themonitoring range and the time (i.e. the imaging time) at which the videowas captured, to the person identification information analysis unit 12.The video acquisition unit 11 is realized, for example, by an imagingdevice such as a camera.

Although the target object identifying device depicted in FIG. 1 as anexample includes only one video acquisition unit 11, the number of videoacquisition units 11 is not limited to one, and may be two or more. Therespective imaging ranges of the video acquisition units 11 may partlyoverlap each other, and may have no overlap with each other. In the casewhere the target object identifying device includes a plurality of videoacquisition units 11, each video acquisition unit 11 may also outputinformation (hereafter referred to as “imaging device identificationinformation”) for identifying the video acquisition unit 11 whenoutputting the acquired video to the person identification informationanalysis unit 12.

In this exemplary embodiment, the video acquisition unit 11 ispreferably installed in a facility to be monitored so that a range whichparticularly requires observation can be captured. The video acquisitionunit 11 does not need to be installed so as to capture the entire rangeof the facility.

The person identification information analysis unit 12 analyzes amonitoring target shown in the video captured by each video acquisitionunit 11. In detail, upon detecting a monitoring target in the video, theperson identification information analysis unit 12 extracts information(hereafter referred to as “monitoring target imaging information”)including the identification information (hereafter referred to as“monitoring target identification information”) of the monitoring targetand the imaging time at which the monitoring target was captured.

The contents of the monitoring target identification informationextracted by the person identification information analysis unit 12 areset beforehand depending on the type of monitoring targets. As anexample, in the case where the monitoring targets are persons, theperson identification information analysis unit 12 may, upon detecting aperson in the video, extract the person's face image as the monitoringtarget identification information. As another example, in the case wherethe monitoring targets are cars, the person identification informationanalysis unit 12 may, upon detecting a car in the video, extract thecar's registration plate image as the monitoring target identificationinformation.

The information extracted by the person identification informationanalysis unit 12 is not limited to the face image or the carregistration plate image. Any information for identifying the monitoringtarget may be extracted. Monitoring target identification informationused for identifying a person may be referred to as “personidentification information”. Since the method of extracting specificidentification information from a target image is widely known, itsdetailed description is omitted here.

In the case where no explicit imaging device identification informationis provided from each video acquisition unit 11, the personidentification information analysis unit 12 may, upon receiving videofrom each video acquisition unit 11, assign imaging deviceidentification information according to the video acquisition unit 11from which the video has been received.

The person identification information management unit 13 stores theinformation analyzed by the person identification information analysisunit 12, in the person identification information storage unit 18. Theperson identification information management unit 13 also extractsnecessary information from the person identification information storageunit 18 and provides the extracted information to the person informationmanagement unit 16, in response to a request from the person informationmanagement unit 16.

The person identification information storage unit 18 stores theinformation analyzed by the person identification information analysisunit 12. In detail, the person identification information storage unit18 stores an identifier for identifying individual monitoring targetimaging information, and the monitoring target identificationinformation and imaging time included in the monitoring target imaginginformation, in association with each other.

FIG. 2 is an explanatory diagram depicting an example of informationstored in the person identification information storage unit 18. FIG. 2depicts an example of information stored in the person identificationinformation storage unit 18 in the case where the monitoring targets arepersons. In the example depicted in FIG. 2, a person image ID is used asthe identifier for identifying the monitoring target imaginginformation, and person identification information is used as themonitoring target identification information. The person identificationinformation storage unit 18 may also store the camera name (e.g. imagingdevice identification information) of the camera that acquired the videoand the person image detected by the person identification informationanalysis unit 12, as depicted in FIG. 2 as an example.

The person information management unit 16 stores information matched bythe person identification information matching unit 15, in the personinformation storage unit 19. The person information management unit 16also extracts necessary information from the person information storageunit 19 and provides the extracted information to the long stayerestimation unit 14, in response to a request from the long stayerestimation unit 14.

The person information storage unit 19 stores each identified monitoringtarget and the imaging times of the monitoring target. FIG. 3 is anexplanatory diagram depicting an example of information stored in theperson information storage unit 19. FIG. 3 depicts an example ofinformation stored in the person information storage unit 19 in the casewhere the monitoring targets are persons. In the example depicted inFIG. 3, the person information storage unit 19 stores, for each piece ofperson identification information (person ID), the identifier (personimage ID) of the monitoring target imaging information from which theperson has been extracted, the imaging count of the person, the earliesttime (hereafter referred to as “earliest imaging time”) at which theperson was captured, and the latest time (hereafter referred to as“latest imaging time”) at which the person was captured, in associationwith each other.

The form in which the imaging times of each identified monitoring targetare stored in the person information storage unit 19 is not limited tothat depicted in FIG. 3 as an example. The person information storageunit 19 may store each individual imaging time of the identifiedmonitoring target. FIG. 4 is an explanatory diagram depicting anotherexample of information stored in the person information storage unit 19.FIG. 4 depicts an example of information stored in the personinformation storage unit 19 in the case where the monitoring targets arepersons, too.

In the example depicted in FIG. 4, the person information storage unit19 stores, for each piece of person identification information (personID), the identifier (person image ID) of the monitoring target imaginginformation from which the person has been extracted and the imagingtime, in association with each other. The person information storageunit 19 may also store, in association with the identifier (person imageID) of the monitoring target imaging information, the camera name (e.g.imaging device identification information) of the camera that acquiredthe video and the probability (likelihood) of the monitoring targetestimated from the acquired video, as depicted in FIG. 4 as an example.

The person identification information matching unit 15 matchesmonitoring target identification information, and identifies monitoringtargets estimated to be the same. For example, in the case where faceimages of monitoring targets are extracted as monitoring targetidentification information, the person identification informationmatching unit 15 may match face images and identify persons estimated tobe the same.

In detail, the person identification information matching unit 15matches the monitoring target identification information included in themonitoring target imaging information stored in the personidentification information storage unit 18 against the monitoring targetidentification information of each person stored in the personinformation storage unit 19, and determines whether or not the sameperson is stored in the person information storage unit 19.

Suppose the person identification information storage unit 18 stores theinformation depicted in FIG. 3 as an example. In the case of determiningthat the same person is stored in the person information storage unit19, the person identification information matching unit 15 compares theimaging time included in the monitoring target imaging information withthe earliest imaging time and latest imaging time of the person storedin the person information storage unit 19.

In the case where the imaging time is earlier than the earliest imagingtime, the person identification information matching unit 15 requeststhe person information management unit 16 to update the earliest imagingtime of the person with the imaging time. In the case where the imagingtime is later than the latest imaging time, the person identificationinformation matching unit 15 requests the person information managementunit 16 to update the latest imaging time of the person with the imagingtime. In the case where the imaging time is later than or equal to theearliest imaging time and earlier than or equal to the latest imagingtime, the person identification information matching unit 15 does notperform the imaging time update process. The person identificationinformation matching unit 15 also requests the person informationmanagement unit 16 to increase the imaging count of the person by 1.

In the case of determining that the same person is not stored in theperson information storage unit 19, on the other hand, the personidentification information matching unit 15 requests the personinformation management unit 16 to newly add the person to the personinformation storage unit 19 and set the earliest imaging time and thelatest imaging time to the imaging time. The person identificationinformation matching unit 15 also requests the person informationmanagement unit 16 to set the imaging count of the person to 1.

Suppose the person identification information storage unit 18 stores theinformation depicted in FIG. 4 as an example. In the case where the sameperson is stored in the person information storage unit 19, the personidentification information matching unit 15 requests the personinformation management unit 16 to store information associating theperson image ID and the imaging time with each other, in the personinformation storage unit 19 in association with the person ID of theperson determined to be the same.

In the case of determining that the same person is not stored in theperson information storage unit 19, on the other hand, the personidentification information matching unit 15 requests the personinformation management unit 16 to assign a new person ID and storeinformation associating the person image ID and the imaging time withthe person ID in the person information storage unit 19.

In the case where the person identification information storage unit 18stores the information depicted in FIG. 4 as an example, the personidentification information matching unit 15 may request the personinformation management unit 16 to store the camera name (e.g. imagingdevice identification information) of the camera that captures thevideo, in the person information storage unit 19 together with theimaging time.

When matching monitoring target identification information, the personidentification information matching unit 15 may calculate theprobability (likelihood) of monitoring targets estimated to be the same.For example, suppose the person identification information matching unit15 performs matching for a face image, and calculates the probability ofdetermining the person as person A at 0.8, the probability ofdetermining the person as person B at 0.3, and the probability ofdetermining the person as person C at 0.2. In this case, the personidentification information matching unit 15 may request the personinformation management unit 16 to store, for the person ID of eachperson for which the probability has been determined, informationassociating the person image ID, the imaging time, and the calculatedprobability with each other in the person information storage unit 19.

Since the method of comparing images and determining whether or not theymatch and the method of calculating the probability of the match arewidely known, their detailed description is omitted here.

The long stayer estimation unit 14 identifies a desired target objectfrom captured monitoring targets, using the imaging times included inthe monitoring target imaging information of each monitoring targetestimated to be the same. In detail, the long stayer estimation unit 14identifies, as the target object, a monitoring target whose frequency ofmonitoring target imaging information with the imaging time included ina predetermined period is greater than or equal to a predeterminedthreshold or whose time width between imaging times in the predeterminedperiod is greater than or equal to a predetermined time. Thepredetermined period is also referred to as “analysis time width”.

The “time width between imaging times” mentioned here means thedifference between any two imaging times selected in the predeterminedperiod. In this exemplary embodiment, the “time width between imagingtimes” can be regarded as the width between any imaging times among theimaging times of the monitoring target captured in the predeterminedperiod. Moreover, the “frequency of monitoring target imaginginformation with the imaging time included in a predetermined period”can be regarded as the frequency of imaging of the monitoring target inthe predetermined period.

The reason for identifying the monitoring target that meets such acondition as the target object is as follows. In the followingdescription, the case of identifying an unusually long stayer in astation is used as an example. The location subjected to the process ofidentifying an unusually long stayer is, however, not limited to astation. The location may be, for example, any range where staying for along time is determined to be unusual, such as a predetermined area of abuilding or a commercial facility.

As an example, a criminal such as a pickpocket, a luggage lifter, or amolester is assumed to stay in a certain area for an unusually longtime, unlike an ordinary passenger. In other words, an ordinarypassenger typically moves from a ticket gate to a platform and then getson a train. Such a passenger who moves in one direction is caught by acamera only for a limited time, and is unlikely to be caught by the samecamera for a long time. Hence, a person who stays in a given area in thestation for a long time can be regarded as an unusually long stayer.

As another example, in the case where a plurality of cameras areinstalled far from each other, even when it takes time to capture atarget person by these cameras, his or her behavior seems to be usual,given the movement time. In the case where a plurality of cameras areinstalled near each other, on the other hand, it seems unusual to taketime to capture a target person by these cameras. Hence, a personcaptured by the cameras over the time width defined between the camerascan be regarded as an unusually long stayer.

An unusually long stayer can also be determined depending on theproperty of the range captured by a camera. For example, regardingmeeting spots or near benches, even when a person stays in one suchplace for a long time, his or her behavior does not seem to beparticularly unusual. When a person wanders from one such place toanother, however, his or her behavior seems to be unusual. Hence, aperson whose frequency of being captured by a plurality of camerasexceeds a predetermined value can also be regarded as an unusually longstayer.

The long stayer estimation unit 14 estimates each unusually long stayeras defined above, and extracts persons who seem to be suspicious, as agray list. Note that the unusually long stayer is not limited to theabove-defined monitoring targets. For example, in the case where camerasare installed at the entrance and the exit, if a person is caught onlyby one camera, his or her behavior seems to be unusual, and accordinglythe person may be defined as an unusually long stayer.

A monitoring target whose time width between imaging times is extremelylong may not be an unusually long stayer. For example, in the case ofmonitoring persons in a station, there is a possibility that the sameperson is captured by a camera around the start of office hours andaround the end of office hours. The movement of such a person seems tobe usual. The long stayer estimation unit 14 may accordingly exclude anymonitoring target whose time width between imaging times is extremelylong, from the target object. To exclude such a monitoring target, anappropriate analysis time width may be set beforehand depending on theproperty of monitoring targets.

The long stayer estimation unit 14 may identify not only a person butalso an object such as a car that is parked for an unusually long time.In this case, the object staying for an unusually long time can beregarded as an unusually long staying object. The following describes amethod whereby the long stayer estimation unit 14 identifies, as adesired target object, an unusually long stayer or an unusually longstaying object from monitoring targets.

A method whereby the long stayer estimation unit 14 identifies thetarget object based on the time width between imaging times in thepredetermined period is described first. The long stayer estimation unit14 may identify, as the target object, a monitoring target whose timewidth between the earliest imaging time and the latest imaging time,among times widths between imaging times in the predetermined period, isgreater than or equal to a predetermined threshold specified by adetermination condition. The predetermined threshold is hereafter alsoreferred to as “long stay determination time”.

This is based on the assumption that a person whose difference betweenthe earliest imaging time and the latest imaging time (i.e. the maximumimaging time interval) is extremely long is an unusually long stayer.

FIG. 5 is an explanatory diagram depicting an example of the operationof determining the stay time of each monitoring target. In the exampledepicted in FIG. 5, persons u1 to u10 as monitoring targets have beenidentified in the direction of the time axis, from video captured by twoimage acquisition units 11 a and 11 b. The period between two dottedlines is the analysis time width, and the period between long dashedlines is the long stay determination time. The range for identifying atarget object, such as 3 hours or 1 day, is designated as the analysistime width, and the time used to determine unusual stay, such as 30minutes, is designated as the long stay determination time.

For example, regarding persons u3 and u4 estimated to be the sameperson, the width between imaging times is shorter than the long staydetermination time. Accordingly, the long stayer estimation unit 14 doesnot determine person u3 (u4) as an unusually long stayer. Likewise,regarding persons u5 and u8 estimated to be the same person, though theimages were captured by the different video acquisition units, the widthbetween the imaging times is shorter than the long stay determinationtime. Accordingly, the long stayer estimation unit 14 does not determineperson u5 (u8) as an unusually long stayer.

Regarding persons u2, u6, u7, and u9 estimated to be the same person, onthe other hand, the difference between the earliest imaging time and thelatest imaging time is longer than the long stay determination time.Accordingly, the long stayer estimation unit 14 determines person u2(u6, u7, u9) as an unusually long stayer.

One overall determination condition for specifying the long staydetermination time may be set regardless of the number of videoacquisition units 11. In this case, the determination condition is acondition for determining the stay time of each monitoring target in theentire facility in which the video acquisition units 11 are installed.

Alternatively, the determination condition for specifying the long staydetermination time may be set for each video acquisition unit 11. Inthis case, the determination condition is a condition for determiningthe stay time of each monitoring target captured in the imaging range ofeach video acquisition unit 11.

Alternatively, the determination condition for specifying the long staydetermination time may be set between the video acquisition units 11. Inthis case, the determination condition is a condition for determining,when each monitoring target moved between the imaging ranges of theplurality of video acquisition units 11, the stay time between theimaging ranges.

FIG. 6 is an explanatory diagram depicting an example of determinationconditions for specifying time widths. A threshold of the time width setfor each video acquisition unit 11 and a threshold of the time width setbetween the video acquisition units 11 are depicted in the example inFIG. 6. In the example in FIG. 6, the acceptable stay time in theimaging range of camera 1 is 10 seconds, and the acceptable stay timebetween the imaging ranges of cameras 1 and 2 is 50 seconds.

Another method whereby the long stayer estimation unit 14 identifies thetarget object based on the time width between imaging times in thepredetermined period is described next. The long stayer estimation unit14 may identify, as the target object, a monitoring target whose timewidth between successive imaging times, among times widths betweenimaging times in the predetermined period, is greater than or equal to apredetermined threshold specified by a determination condition. This isbased on the assumption that a person whose time width between imagingtimes by different video acquisition units 11 is extremely long is anunusually long stayer.

One overall determination condition for specifying the condition of thetime width between successive imaging times may be set regardless of thenumber of video acquisition units 11. In this case, the determinationcondition is a condition for determining the time of movement of eachmonitoring target in the facility in which the video acquisition units11 are installed.

Alternatively, the determination condition for specifying the conditionof the time width between successive imaging times may be set for eachvideo acquisition unit 11. In this case, the determination condition isa condition for determining the acceptable stay time of each monitoringtarget according to the property of the imaging range of each videoacquisition unit 11.

Alternatively, the determination condition for specifying the conditionof the time width between successive imaging times may be set betweenthe video acquisition units 11. In this case, the determinationcondition is a condition for determining the time of movement of eachmonitoring target between the imaging ranges of the plurality of videoacquisition units 11. For example, in the case where the time widthbetween the imaging times by the video acquisition units 11 installednear each other is extremely long, the person seems to be an unusuallylong stayer.

The following describes a method of identifying the target object basedon the determination condition for specifying the condition of the timewidth between successive imaging times, with reference to FIGS. 6 and 7.FIG. 7 is an explanatory diagram depicting a specific example ofinformation stored in the person information storage unit 19. Here,three video acquisition units 11 are referred to as camera 1, camera 2,and camera 3. Consider the case where the condition depicted in FIG. 6as an example has been set as the determination condition for specifyingthe time width.

Suppose camera 1 captured video identified by person image ID 1 at00:00:00, and then camera 2 captured video identified by person image ID2 at 00:00:30. In the example depicted in FIG. 6, the condition set asthe time width between cameras 1 and 2 is 50 seconds. Since thedifference between the two imaging times is 30 seconds, the long stayerestimation unit 14 does not identify the monitoring target as the targetobject.

Further, suppose camera 3 captured video identified by person image ID 3at 00:00:45. In the example depicted in FIG. 6, the condition set as thetime width between cameras 2 and 3 is 40 seconds. Since the differencebetween the two imaging times is 15 seconds, the long stayer estimationunit 14 does not identify the monitoring target as the target object.

Suppose, on the other hand, camera 3 captured the video identified byperson image ID 3 at 00:02:00. Since the difference between the twoimaging times is 90 seconds, the long stayer estimation unit 14identifies the monitoring target as the target object.

Although the above describes the case where the determination conditionfor specifying the condition of the long stay determination time is setbeforehand, the determination condition may be dynamically generatedbased on the imaging times of monitoring targets not identified as thetarget object.

The following describes examples of the determination conditiongeneration method. The person information management unit 16 may keepstatistics about the stay time of each monitoring target whose stay timeis shorter than the long stay determination time, to calculate the longstay determination time.

In detail, the person information management unit 16 may calculate, fromthe information of each monitoring target subjected to the statistics,the stay time mean in the maximum imaging time interval or the timeinterval between the video acquisition units 11, and calculate the timewhich is a constant multiple of (e.g. double) the stay time mean as thelong stay determination time.

As an alternative, the person information management unit 16 maycalculate, from the information of each monitoring target subjected tothe statistics, the stay time mean and variance in the maximum imagingtime interval or the time interval between the video acquisition units11, and calculate a threshold of the time interval in which apredetermined proportion (e.g. 5%) of people are determined as longstayers, as the long stay determination time.

The method of calculating the long stay determination time is notlimited to these methods. For example, the movement speed of eachmonitoring target in video captured by one video acquisition unit 11 maybe measured, with the long stay determination time being calculatedbased on the movement speed and the distance between the videoacquisition units 11.

Dynamically calculating the long stay determination time in this wayenables use of an appropriate determination condition according to theactual situation, as a result of which the target object can beidentified more appropriately from monitoring targets.

A method whereby the long stayer estimation unit 14 identifies thetarget object based on the imaging frequency of each monitoring targetin the predetermined period is described next. The long stayerestimation unit 14 may identify, as the target object, a monitoringtarget whose frequency of monitoring target imaging information with theimaging time included in the predetermined period is greater than orequal to a predetermined threshold. This is based on the assumption thata monitoring target captured extremely frequently is an unusually longstayer.

FIG. 8 is an explanatory diagram depicting an example of the operationof capturing a monitoring target by a plurality of video acquisitionunits 11. In the example depicted in FIG. 8, video acquisition units 11a, 11 b, 11 c, and 11 d respectively capture areas A, B, C, and D.

When any person is detected from captured video, the personidentification information of each person is extracted and stored in theperson identification information storage unit 18. In the exampledepicted in FIG. 8, person u11 was captured in video of each of areas A,B, C, and D. Accordingly, four entries of the person identificationinformation relating to person u11 are registered in the personidentification information storage unit 18. Based on this registrationfrequency, the long stayer estimation unit 14 identifies person u11 frommonitoring target persons, as the target object. The result output unit17 outputs the identified target object to, for example, an outputresult storage unit 20.

One overall determination condition for specifying the condition of thefrequency may be set regardless of the number of video acquisition units11. In this case, the determination condition is a condition fordetermining the imaging frequency of each monitoring target in theentire facility in which the video acquisition units 11 are installed.

Alternatively, the determination condition for specifying the conditionof the frequency may be set for each video acquisition unit 11. In thiscase, the determination condition is a condition for determining theimaging frequency of each monitoring target in the imaging range of eachvideo acquisition unit 11.

Alternatively, the determination condition for specifying the conditionof the frequency may be set for each group of any video acquisitionunits 11. In this case, the determination condition is a condition fordetermining the imaging frequency of each monitoring target in theimaging ranges of the video acquisition units 11 belonging to the group.

For example, in the case where a monitoring target was captured aplurality of times in one meeting spot, the monitoring target does notseem to be an unusually long stayer. In the case where a monitoringtarget was captured a plurality of times in a plurality of meetingspots, however, the monitoring target seems to be an unusually longstayer. Such an unusually long stayer can be identified by setting thecondition relating to any video acquisition units 11 that are groupedtogether.

In the case where the person identification information matching unit 15calculates the probability (likelihood) of monitoring targets estimatedto be the same, the long stayer estimation unit 14 may identify thetarget object using the frequency calculated based on the likelihood.For example, suppose two entries of information of a monitoring targetare registered in the person information storage unit 19. In the casewhere the likelihood of the monitoring target is 0.5 in each entry, thelong stayer estimation unit 14 may determine the frequency as 1(=0.5×2), and identify the target object.

In the case where the person identification information matching unit 15calculates the likelihood, the long stayer estimation unit 14 mayperform the estimation process only on monitoring target imaginginformation with the likelihood being greater than or equal to apredetermined value. This is equally applicable when the long stayerestimation unit 14 identifies the target object based on the time widthbetween imaging times in the predetermined period. In detail, the longstayer estimation unit 14 may calculate the time width between imagingtimes included in monitoring target imaging information with thelikelihood being greater than or equal to the predetermined value, for amonitoring target.

The long stayer estimation unit 14 generates a list of images, imagingdevice identification information, imaging times, and the like of theidentified target object, and provides the list to the result outputunit 17.

The result output unit 17 outputs the identified target object by agiven method. For example, the result output unit 17 may store theoutput result in the output result storage unit 20. The result outputunit 17 may also extract a common monitoring target extracted indifferent times or different days, from information stored in the outputresult storage unit 20. Since such a target object particularly needs tobe observed, the result output unit 17 may, for example, output theinformation of the person as a habitual unusually long stayer.

The person identification information analysis unit 12, the personidentification information management unit 13, the long stayerestimation unit 14, the person identification information matching unit15, the person information management unit 16, and the result outputunit 17 are realized, for example, by a CPU of a computer operatingaccording to a program (target object identifying program). Forinstance, the program may be stored in a storage unit (not depicted) inthe target object identifying device, with the CPU reading the programand, according to the program, operating as the person identificationinformation analysis unit 12, the person identification informationmanagement unit 13, the long stayer estimation unit 14, the personidentification information matching unit 15, the person informationmanagement unit 16, and the result output unit 17.

Alternatively, the person identification information analysis unit 12,the person identification information management unit 13, the longstayer estimation unit 14, the person identification informationmatching unit 15, the person information management unit 16, and theresult output unit 17 may each be realized by dedicated hardware. Theperson identification information storage unit 18 and the personinformation storage unit 19 are realized, for example, by a magneticdisk or the like.

The following describes the operation of the target object identifyingdevice in this exemplary embodiment. FIG. 9 is a flowchart depicting anexample of the operation of the target object identifying device in thisexemplary embodiment.

First, the video acquisition unit 11 acquires video of its imaging range(step S11), and outputs the captured video to the person identificationinformation analysis unit 12. Upon detecting a monitoring target fromthe received video, the person identification information analysis unit12 extracts the identification information of the monitoring target(step S12), and stores the extracted monitoring target identificationinformation in the person identification information storage unit 18(step S13).

The person identification information matching unit 15 matches themonitoring target identification information stored in the personidentification information storage unit 18, against the monitoringtarget identification information of each monitoring target stored inthe person information storage unit 19 (step S14). In the case where thesame monitoring target is found (step S15: Yes), the person informationmanagement unit 16 updates the imaging time of the monitoring target(step S16). In the case where the same monitoring target is not found(step S15: No), on the other hand, the person information managementunit 16 stores the monitoring target in the person information storageunit 19 as a new monitoring target, together with the imaging time (stepS17).

The long stayer estimation unit 14 identifies a desired target objectfrom monitoring targets, using the imaging times included in themonitoring target imaging information of the identified same persons(step S18). The long stayer estimation unit 14 identifies the targetobject based on the time width between imaging times or the frequency ofimaging times. The result output unit 17 outputs the identified targetobject (step S19).

As described above, according to this exemplary embodiment, the personidentification information analysis unit 12 analyzes a monitoring targetshown in video, and extracts the corresponding monitoring target imaginginformation including the monitoring target identification informationand the imaging time. The person identification information matchingunit 15 matches the monitoring target identification information, andidentifies the monitoring target estimated to be the same. The longstayer estimation unit 14 identifies a desired target object frommonitoring targets, using the imaging times included in the monitoringtarget imaging information of each identified monitoring target.

For example, the long stayer estimation unit 14 identifies, as thetarget object, a monitoring target whose frequency of monitoring targetimaging information with the imaging time included in the predeterminedperiod is greater than or equal to the predetermined threshold or whosetime width between imaging times in the predetermined period is greaterthan or equal to the predetermined time. In this way, even in the casewhere the monitoring range is wide or crowded, a target object presentin the range for an unusually long time can be identified frommonitoring targets.

For example, suppose the target object identifying device in thisexemplary embodiment is used in a station. First, the personidentification information analysis unit 12 acquires information foridentifying a face or a person, from video captured by one or more videoacquisition units 11 installed in the station. The person identificationinformation matching unit 15 matches the information for identifying theperson against information acquired in different times or by a pluralityof cameras, to calculate the time width or frequency of imaging of thesame person. Thus, a list of imaging times can be generated for eachmonitoring target, even in the case where information of any suspiciousperson is not provided beforehand.

The long stayer estimation unit 14 finds an unusually long stayer in thefacility different from ordinary users, using the time width, thefrequency, or the like. There is accordingly no need to install camerasto cover the entire monitoring target area and, even in the case wherepart of monitoring targets cannot be captured in a crowded situation,monitoring targets can be identified from other imaging times.

The following gives an overview of the present invention. FIG. 10 is ablock diagram schematically depicting the target object identifyingdevice according to the present invention. The target object identifyingdevice according to the present invention includes: monitoring targetmatching means 71 (e.g. the person identification information matchingunit 15) which matches monitoring targets shown in video captured by oneor more imaging devices (e.g. the video acquisition unit 11), andidentifies monitoring targets estimated to be the same monitoringtarget, as an identified monitoring target; and target objectidentifying means 72 (e.g. the long stayer estimation unit 14) whichidentifies a desired target object from one or more identifiedmonitoring targets, using imaging times of each of the one or moreidentified monitoring targets.

With such a structure, even in the case where the monitoring range iswide or crowded, a target object present in the range for an unusuallylong time can be identified from monitoring targets.

The target object identifying means 72 may identify, as the targetobject, an identified monitoring target whose frequency of imaging timesin a predetermined period (e.g. the analysis time width) is greater thanor equal to a predetermined frequency or whose time width betweenimaging times in the predetermined period is greater than or equal to apredetermined time (e.g. the long stay determination time).

The target object identifying means 72 may identify, as the targetobject, an identified monitoring target whose time width between anearliest imaging time and a latest imaging time, among time widthsbetween imaging times in a predetermined period, is greater than orequal to a predetermined threshold (e.g. the long stay determinationtime) specified by a determination condition.

The target object identifying means 72 may identify the target objectbased on the determination condition (e.g. the determination conditiondepicted in FIG. 6) for specifying, for each imaging device, a conditionof the time width between the earliest imaging time and the latestimaging time.

The target object identifying device may include determination conditiongeneration means (e.g. the person information management unit 16) whichgenerates the determination condition for specifying a condition of thetime width between the earliest imaging time and the latest imagingtime, based on imaging times of monitoring targets not identified as thetarget object. Dynamically calculating the determination condition inthis way enables use of an appropriate determination condition accordingto the actual situation, as a result of which the target object can beidentified more appropriately from monitoring targets.

The target object identifying means 72 may identify, as the targetobject, an identified monitoring target whose time width betweensuccessive imaging times, among time widths between imaging times in apredetermined period, is greater than or equal to a predeterminedthreshold specified by a determination condition.

The target object identifying means 72 may identify, as the targetobject, the identified monitoring target whose time width between thesuccessive imaging times meets the determination condition betweenimaging devices.

The target object identifying means 72 may identify the target objectbased on a determination condition for specifying, for each imagingdevice, a frequency of imaging times of an identified monitoring targetincluded in a predetermined period.

The target object identifying means 72 may identify the target objectbased on a determination condition for specifying, for eachpredetermined imaging device group, the frequency of the imaging timesof the identified monitoring target included in the predeterminedperiod.

The monitoring target matching means 71 may calculate a likelihood ofthe monitoring targets estimated to be the same monitoring target, andthe target object identifying means 72 may identify the target object,based on the frequency calculated based on the likelihood.

The monitoring target matching means 71 may match face images extractedfrom the video captured by the one or more imaging devices, and identifymonitoring targets estimated to be the same person.

The monitoring target matching means 71 may extract information(monitoring target identification information) for identifying eachmonitoring target, before matching monitoring targets shown in thevideo. In detail, the monitoring target matching means 71 may executethe process performed by the person identification information analysisunit 12 in Exemplary Embodiment. In the case where the monitoringtargets can be matched without using monitoring target identificationinformation, however, the explicit extraction process for monitoringtarget identification information is not necessarily required. Thefollowing describes a target object identifying device including thestructure for extracting monitoring target identification information.

FIG. 11 is another block diagram schematically depicting the targetobject identifying device according to the present invention. The targetobject identifying device according to the present invention includes:monitoring target imaging information analysis means 81 (e.g. the personidentification information analysis unit 12) which analyzes monitoringtargets (e.g. persons, cars) shown in video captured by one or moreimaging devices (e.g. the video acquisition unit 11), and extractsmonitoring target imaging information that includes monitoring targetidentification information (e.g. face image, car registration plate)used for identifying each monitoring target and an imaging time at whichthe monitoring target was captured; monitoring target matching means 82(e.g. the person identification information matching unit 15) whichmatches the monitoring target identification information, and identifiesmonitoring targets estimated to be the same monitoring target; andtarget object identifying means 83 (e.g. the long stayer estimation unit14) which identifies a desired target object (e.g. an unusually longstayer) from captured monitoring targets, using imaging times includedin monitoring target imaging information of each identified monitoringtarget.

With such a structure, too, even in the case where the monitoring rangeis wide or crowded, a target object present in the range for anunusually long time can be identified from monitoring targets.

The target object identifying means 83 may identify, as the targetobject, a monitoring target whose frequency of monitoring target imaginginformation with an imaging time included in a predetermined period(e.g. the analysis time width) is greater than or equal to apredetermined threshold or whose time width between imaging times in thepredetermined period is greater than or equal to a predetermined time(e.g. the long stay determination time).

In detail, the target object identifying means 83 may identify, as thetarget object, a monitoring target whose time width between an earliestimaging time and a latest imaging time, among time widths betweenimaging times in a predetermined period, is greater than or equal to apredetermined threshold (e.g. the long stay determination time)specified by a determination condition.

The monitoring target imaging information analysis means 81 may extractthe monitoring target imaging information including imaging deviceidentification information used for identifying an imaging device thatcaptured the monitoring target. Here, the target object identifyingmeans 83 may identify the target object based on the determinationcondition (e.g. the determination condition depicted in FIG. 6) forspecifying, for each imaging device, a condition of the time widthbetween the earliest imaging time and the latest imaging time.

The target object identifying device may include determination conditiongeneration means (e.g. the person information management unit 16) whichgenerates the determination condition for specifying a condition of thetime width between the earliest imaging time and the latest imagingtime, based on imaging times of monitoring targets not identified as thetarget object.

The target object identifying means 83 may identify, as the targetobject, a monitoring target whose time width between successive imagingtimes, among time widths between imaging times in a predeterminedperiod, is greater than or equal to a predetermined threshold specifiedby a determination condition.

The monitoring target imaging information analysis means 81 may extractthe monitoring target imaging information including imaging deviceidentification information used for identifying an imaging device thatcaptured the monitoring target. The target object identifying means 83may identify, as the target object, the monitoring target whose timewidth between the successive imaging times in the monitoring targetimaging information meets the determination condition between imagingdevices.

The monitoring target imaging information analysis means 81 may extractthe monitoring target imaging information including imaging deviceidentification information used for identifying an imaging device thatcaptured the monitoring target. The target object identifying means 83may identify the target object based on a determination condition forspecifying, for each imaging device, a condition of the frequency of themonitoring target imaging information with the imaging time included inthe predetermined period.

The target object identifying means 83 may identify the target objectbased on a determination condition for specifying, for eachpredetermined imaging device group, a condition of the frequency of themonitoring target imaging information with the imaging time included inthe predetermined period.

The monitoring target matching means 82 may calculate a likelihood(probability) of the monitoring targets estimated to be the samemonitoring target. The target object identifying means 83 may identifythe target object using the frequency calculated based on thelikelihood.

The monitoring target imaging information analysis means 81 may extracta face image of each monitoring target as the monitoring targetidentification information. The monitoring target matching means 82 maymatch face images, and identify monitoring targets estimated to be thesame person. With such a structure, a person staying in the monitoringrange for an unusually long time can be identified.

Although the present invention has been described with reference to theforegoing exemplary embodiment and examples, the present invention isnot limited to the foregoing exemplary embodiment and examples. Variouschanges understandable by those skilled in the art within the scope ofthe present invention can be made to the structures and details of thepresent invention.

This application claims priority based on Japanese Patent ApplicationNo. 2013-072178 filed on Mar. 29, 2013, the disclosure of which isincorporated herein in its entirety.

INDUSTRIAL APPLICABILITY

The present invention is suitable for use in, for example, a videoanalysis system for capturing persons in a specific facility by one ormore surveillance cameras and analyzing the captured video toautomatically find any long stayer.

REFERENCE SIGNS LIST

11, 11 a to 11 d video acquisition unit

12 person identification information analysis unit

13 person identification information management unit

14 long stayer estimation unit

15 person identification information matching unit

16 person information management unit

17 result output unit

18 person identification information storage unit

19 person information storage unit

20 output result storage unit

u1 to u11 person

The invention claimed is:
 1. A target object identifying devicecomprising: at least one memory configured to store instructions; and atleast one processor configured to execute the instructions to perform:matching monitoring targets shown in a video; determining, as a targetobject, a monitoring target whose time width between an earliest imagingtime and a latest imaging time in a predetermined period, is greaterthan or equal to a long term stay determination time; generating adetermination condition for specifying the long term stay determinationtime based on statistics of stay time of monitoring targets notidentified as the target object; calculating, as the statistics, a staytime mean based on maximum imaging time interval or time interval of theimaging time of the monitoring targets not identified as the targetobject, and calculating a time which is a constant multiple of the staytime mean as the long term stay determination time; and determining, asthe target object, the monitoring target whose time width is greaterthan or equal to the calculated long term stay determination time byusing the calculated long term stay determination time as a thresholdvalue.
 2. The target object identifying device according to claim 1,further comprising a multiple of imaging devices capturing the video. 3.The target object identifying device according to claim 2, wherein thelong stay determination time is calculated based on the movement speedof each monitoring target subjected to the statistics and the distancebetween the imaging devices.
 4. The target object identifying deviceaccording to claim 1, wherein the at least one processor is furtherconfigured to perform: calculating, as the statistics, a stay time meanand variance based on maximum imaging time interval or time interval ofthe imaging time of the monitoring targets not identified as the targetobject, and calculating a threshold of the time interval in which apredetermined proportion of people are determined as long stayers, asthe long term stay determination time.
 5. A target object identifyingmethod comprising: matching monitoring targets shown in a video;determining, as a target object, a monitoring target whose time widthbetween an earliest imaging time and a latest imaging time in apredetermined period, is greater than or equal to a long term staydetermination time; generating a determination condition for specifyingthe long term stay determination time based on statistics of stay timeof monitoring targets not identified as the target object; calculating,as the statistics, a stay time mean based on maximum imaging timeinterval or time interval of the imaging time of the monitoring targetsnot identified as the target object, and calculating a time which is aconstant multiple of the stay time mean as the long term staydetermination time; and determining, as the target object, themonitoring target whose time width is greater than or equal to thecalculated long term stay determination time by using the calculatedlong term stay determination time as a threshold value.
 6. The targetobject identifying method according to claim 5, comprising matching themonitoring targets shown in the video, the video captured by a multipleof imaging devices.
 7. The target object identifying method according toclaim 6, wherein the long stay determination time is calculated based onthe movement speed of each monitoring target subjected to the statisticsand the distance between the imaging devices.
 8. A non-transitoryrecording medium storing a computer program, the computer programcausing a computer to perform: matching monitoring targets shown in avideo; determining, as a target object, a monitoring target whose timewidth between an earliest imaging time and a latest imaging time in apredetermined period, is greater than or equal to a long term staydetermination time; generating a determination condition for specifyingthe long term stay determination time based on statistics of stay timeof monitoring targets not identified as the target object; calculating,as the statistics, a stay time mean based on maximum imaging timeinterval or time interval of the imaging time of the monitoring targetsnot identified as the target object, and calculating a time which is aconstant multiple of the stay time mean as the long term staydetermination time; and determining, as the target object, themonitoring target whose time width is greater than or equal to thecalculated long term stay determination time by using the calculatedlong term stay determination time as a threshold value.
 9. Thenon-transitory recording medium according to claim 8, wherein thecomputer program causes the computer to perform: controlling a multipleof imaging devices to capture the video.
 10. The non-transitoryrecording medium according to claim 9, wherein the long staydetermination time is calculated based on the movement speed of eachmonitoring target subjected to the statistics and the distance betweenthe imaging devices.